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Abstract. In this paper we prove symmetry results for classical 
solutions of nonlinear cooperative elliptic systems in a ball or in an- 
nulus in K , N > 2. More precisely we prove that solutions having 
Morse index j < N are foliated Schwarz symmetric if the nonlin- 
earity is convex and a full coupling condition is satisfied along the 
solution. 



1. Introduction and statement of the results 

We consider a semilinear elliptic system of the type 



1.1^ 



-AU = F(\x\,U) in ft 
U = on on 



where F = (fx, ■ ■ ■ , f m ) is a function belonging to C 1,a ([0, +oo) x 
R m ; R m ) and Q is a bounded domain in R N , m,N > 2. Here U = 
(tlx, . . . , u m ) is a vector valued function in Q. 

It is well known that systems of this type arise in many applications in 
different fields, we refer to [15], [H] for some of these. 

We are interested in studying symmetry properties of classical solu- 
tions of ( II. ip when Q is rotationally symmetric i.e. a ball or an annulus 
centered at the origin in R N . If Q is a ball and the system is cooperative 
then by using the famous "moving plane method" ([21], [ID]) it can be 
proved (see [I], [6], [7], [23] ) that every positive solution U (i.e. Ui > 
in Q for any i — 1, . . . , m) is radial and radially decreasing under the 
additional hypothesis that each fi is nonincreasing with respect to \x\. 
Here we consider solutions of any sign, do not require / to be monotone 
in \x\ and would like to obtain results also in the case of an annulus 
in the same direction of the results obtained in [T7] and [T^] for scalar 
equations. We recall that in [T7] and [19] (see also [11] ) it was proved 
that classical solutions with Morse index less than or equal to iV are 
foliated Schwarz symmetric if the nonlinearity is convex or its first de- 
rivative is convex and Q is a ball or an annulus. 
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Another symmetry result obtained by symmetrization for some partic- 
ular systems, is contained in [TB] . 

Let us give the definition of a foliated Schwarz symmetric function. 

DEFINITION 1.1. Let Q be a rotationally symmetric domain in 
Mr, N > 2. We say that a continuous vector valued function U = 
(ui, . . . ,u m ) : Q — > M m is foliated Schwarz symmetric if each compo- 
nent Ui is foliated Schwarz symmetric with respect to the same vector 
p G M N . In other words there exists a vector p G M N , \p\ = 1, such 

that U(x) depends only on r = \x\ and 9 = arccos - pj and U is 

(componentwise) nonincreasing in 9. 

Remark 1.1. Let us observe that if U is a solution of fll.ip and the 
system satisfies some coupling conditions, as required in Theorem 11.11 
then the foliated Schwarz symmetry of U implies that either U is ra- 
dial or it is strictly decreasing in the angular variable 9. This will be 
deduced by the proof of Theorem 11.11 

In order to state our results we need to define the Morse index for 
solutions of fll.ip and the coupling conditions for the system. However 
they will be restated and commented in Section 2 for general semilinear 
elliptic systems. 

DEFINITION 1.2. i) Let U be a C 2 (fi;M m ) solution of (JTIj) . 
We say that U is linearized stable (or that it has zero Morse 
index) if the quadratic form 



'1.21 



Q V (*;Q)= f [|W| 2 - J F (\x\,V)(*,*)]dx 
Jn 



m m rj r 



du 

i=i *ii=i 



dx > 



for any * = (^i, ... , ifj m ) G C£(fi; R m ) where J F {x, U) is the 
jacobian matrix of F computed at U. 
ii) U has (linearized) Morse index equal to the integer \i = fi(U) > 
1 if /i is the maximal dimension of a subspace of C^fijM" 1 ) 
where the quadratic form is negative definite. 



-(|z|, «i, . . .,u m ) > V (x,u u . . .,u m ) G Q' x 



DEFINITION 1.3. • We say that the system (JIT]) is coopera- 
tive or weakly coupled in an open set Q' C Q if 

for any i, j — 1, . . . , m with % ^ j. 
• We say that the system (II. II) is fully coupled along a solution U 
in an open set Q' C Q if it is cooperative in Q' and in addition 
V/, J C {1, . . . , m} such that 1^0, J^0, / n J = 0, J U J = 
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{1, . . . , m} there exist i G /, jo G J such that 

meas ({a; G Q' : ^-(\x\,U(x)) > 0}) > 
du jo 

As it will be clear in the following sections the previous definition 
reflects the fact that the linearized system at a solution U is weakly or 
fully coupled, according to Definition 12.11 

Let us point out that the last definition is slightly different from the 
usual one stated in the literature, because the "full coupling conditions" 
on the derivatives are required only when computed on the solution 
but on a subset Q'. It is not difficult to see that there are systems 
which are fully coupled along some solution but not fully coupled in 
the usual sense. This was observed and used in [H] and we will give 
some examples in Section 4. 

Let e G S N_1 be a direction, i.e. e G Wt N , |e| = 1, and let us define 
the set 

Q(e) = {x G Q : x ■ e > 0} 

THEOREM 1.1. Let SI be a ball or an annulus in R^, N >2, and let 

U G C 3 ' a (n;lR m ) be a solution of flU]) with Morse index fi(U) < N. 
Moreover assume that: 

i) The system is fully coupled along U in fi(e), for any e G S 1 ^^ 1 . 

ii) For any i,j = 1, . . . m J^(|x|, u%, u m ) is nondecreasing in 
each variable Uk, k = 1, . . . , m, for any \x\ G Q. 

iii) if m > 3 then, for any i G {l,...,m} ; Ui, . . . , u m ) = 
Y.k^i9ik(\x\,Ui,u k ) where g ik G C 1,Q! ([0, +oo) x 1R 2 ). 

Then U is foliated Schwarz symmetric and if U is not radial then it is 
strictly decreasing in the angular variable (see Definition 

Remark 1.2. • Requiring that the system is fully coupled along 
U in Q(e), for any e G S 1 ^ -1 , is an assumption easily satisfied 
by most of the systems encountered in applications (see Section 
4). It is essentially needed to ensure the validity of the strong 
maximum principle in the domains fl(e). 

• The monotonicity hypothesis ii) on the derivatives in The- 
orem 11.11 implies that each /, is convex with respect to each 
variable uj, i, j = 1, . . . , m. 

• The assumption iii), which states that each fi is the sum of 
functions which depend only on ui and one of the other variables 
Uk, is obviously not needed if m = 2. 

Hypotheses ii) and iii) together imply the following condi- 
tions, that will be exploited in the proof: 

a ) fJ(M,ui,---,iv) ( = Efc^fSOl)^!^) if m > 3 ) 
is nondecreasing in Ui for any i = 1, . . . , m, i.e. if U — 
(wi, . . . ,ut,. . . ,u m ) and U = (ui, . . . ,Ui, . . .,u m ) with m < 
Ui then §k(|x|, U) < ^(\x\,U) for any x <E Q. 
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b ) §^(\x\,ui,...,u m )( = ^(\x\,u i ,u j )tfm>3) 

depends only on U{, Uj and it is nondecreasing in Ui, Uj, i.e. 
if U = (wi, . . . , u m ), V = (vi, . . . , v m ) and ui < v h uj < vj, 
then §&(\x\,U) < §jj-(\x\,V) for any i,j = l,...,m for 
any igQ. 

Obviously the previous theorem holds in particular for stable solu- 
tions. However in this for scalar equations, it is not difficult 
to see that the solution is radial, even without the assumption ii) and 
iii). Therefore we have the following 

THEOREM 1.2. If the system is fully coupled along a stable solution 
U in Q, then U is radial. 

We shall see in the proof of Theorem 11.11 that for nonradial Morse 
index one solutions the following conditions hold. 



COROLLARY 1.1. Under the assumptions of Theorem \l.l\ if a so- 
lution U has Morse index one and it is not radial then necessarily 

(1 .S) £ « (r, U(r, 0) = E M (r , U(r , ,))^ (r , ,) 

j = l J j—l 

for any i = 1, . . . , m, whith (r, 9) as in Definition \l.l[ 
In particular if m = 2 then (11. 3p implies that 

(i.4) |^(M, [/(*)) = ^(\x\, u( x )), wxen 

ou 2 au\ 

Remark 1.3. The result of Corollary 11.11 is somewhat surprising be- 
cause it asserts, under the hypotheses of Theorem ll.l[ that for any 
nonradial Morse index one solution another coupling condition, namely 
(ll.3p (and in particular ( 11.4]) if m — 2), must hold along the solution. 
Some consequences of this, for a particular system, will be illustrated 
in Section 4. 



Let us now explain the strategy of the proof of Theorem 11.11 
As in the scalar case the key idea, introduced in [T7], to get the 
symmetry of the solution, exploiting the information on its Morse in- 
dex, is to use convexity assumptions on the nonlinearity to relate the 
linearized operator at the solution with the linear operator that arises 
when considering the difference between the solution and its reflection 
with respect to an hyperplane passing through the origin (see |17j . 
[18] . [19], pj]). To do this, in the scalar case and for bounded domains 
Q, the information on the quadratic form associated to the linearized 
operator at the solution, deduced by its Morse index, is exploited by 
means of the eigenvalues of the linearized operator. Indeed this oper- 
ator is self adjoint and so there is a variational characterization of the 
eigenvalues which links them to the quadratic form. This is partially 
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used also in the unbounded domain case (see [TT]) even if the spectrum 
cannot be easily described, exploiting the eigenvalues of the linearized 
operator in large balls. 

In the case of systems this no longer holds. Indeed if the Jaco- 
bian matrix Jp(\x\,U) is not symmetric the linearized operator Ly 
(see f)3.3p ) is not selfadjoint, even in bounded domains. Then no vari- 
ational characterization of eigenvalues can be exploited and hence the 
scalar case approach cannot be straigtforward followed. 

To bypass this difficulty we associate to the Jacobian matrix Jp(|x|, U) 
its symmetric part \{Jp{\x\, U) + Jp(\x\, U), where J F is the transpose 
of the matrix Jp (see (12. 8p in the linear case), which then gives rise to 
the selfadjoint operator L v (see (13.41) ) whose spectrum can be varia- 
tionally characterized. 

The crucial, simple remark is that the quadratic form associated to 
the linearized operator L v is the same as the quadratic form associated 
to the selfadjoint operator Ljj. 

Therefore the eigenvalues of L v can be exploited to study the sym- 
metry of the solution U, using the information on the Morse index. 
However, though essential in our proofs, the consideration of these 
eigenvalues is not sufficient to reach the conclusion of our theorems, 
but we also need to use the principal eigenvalue of the linearized oper- 
ator, that can be defined for general linear operators (see [3], [5].|22]). 
This can be understood by the fact that the positivity of this eigen- 
value is a necessary and sufficient condition for the (weak) maximum 
principle to hold (see Proposition I2.2p and the maximum principle is 
another key-ingredient in our proofs. 

Obviously if the jacobian matrix Jp(\x\,U) is symmetric then the 
linearized operator is selfadjoint and the classical spectral theory holds, 
in particular the principal eigenvalue coincides with the first eigenvalue 
and all proofs are simpler. 

This happens, for example, when the system is of gradient type, i.e. 
when F = grad g for some scalar function g (see [7]). In this case the 
linearized operator corresponds to the second derivative of a suitable 
associated functional. 

However this is not the case for many interesting systems, like e.g. 
the so called hamiltonian systems (see [7] and the references therein). 

Another important remark about the assumptions of our theorems 
is that, as in previous symmetry results for systems (see [I], [6], [13]). 
coupling conditions on the system are necessary. Indeed it is easy to 
construct counterexamples to the symmetry if they do not hold. 

In particular, since the definition of foliated Schwarz symmetry for 
vector- valued functions requires that all components are foliated Schwarz 
symmetric with respect to the same vector, a strong coupling condition 
is needed. 
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Finally let us make a general comment on symmetry theorems of the 
type described in this paper. 

As for the scalar case, to apply our symmetry results, some information 
on the Morse index of the solution is needed. So the question is: how 
to get it? 

If the system is of gradient type then, often, variational methods, 
used to prove the existence of solutions, also carry information on the 
Morse index (pQ, [7], [H]). A standard example is given by solutions 
obtained by the Mountain Pass theorem. 

If the system is not of this type it could happen that using variational 
methods, one ends up with the second derivative of a functional which 
is strongly indefinite. This is for example the case of the so called 
Hamiltonian systems. 

However this does not mean that solutions do not have finite (lin- 
earized) Morse index, because the linearized operator is not the second 
derivative of such functionals. Then the linearized operators and its 
Morse index could be studied by some other methods, like continua- 
tion techniques. We give some examples in Section 4. 

Since the study of the linearized system is also important to un- 
derstand the orbital stability of the solution and informations on the 
Morse index are useful to get qualitative properties of solutions, we 
believe that further investigation in this direction should be done. 

The outline of the paper is the following. In Section 2 we state and 
prove some important results on the spectral theory and maximum 
principles for linear systems. Most of them are either well known or 
could be easily deduced by known theorems, but we think that is worth 
to recall and collect them together to make the paper self-contained and 
to avoid to give vague references to the reader. 

In Section 3 we show preliminary results on our semilinear system ( 11. ip 
and prove our symmetry results. 

In Section 4 we present a few examples and make further comments. 



2. Preliminaries on linear systems: spectral theory and 



Let Q be any smooth bounded domain in Mr, N > 2, and D amxm 
matrix with bounded entries: 



maximum principles 



(2.1) 




Let us consider the linear elliptic system 



(2.2) 




in f2 

on dVl 
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i.e. 

' -Aiii + duux H h di m u m = fi in fi 



/\u m -\- d m ]U\ -\- • • • -\- d mm u m f m in fi 

k Mi = • • • = u m — on dQ 

where F = (f\, . . . , f m ) E (L 2 (Q)) m , U = {U u . . . , U m ). 

This kind of linear system appears in the linearization of the semilinear 

elliptic system (11.11) . 

DEFINITION 2.1. The system (Q is said to be 

• cooperative or weakly coupled in fi if 

(2.3) <iy < a.e. in fi, whenever i ^ j 

• fully coupled in fi if it is weakly coupled in fi and the following 
condition holds: 

(2.4) 

V/, Jc {l,...,m},I, J=£ 0, in J = 0, JU J= {l,...,m} 

Biq E I , jo E J : meas ({x G fi : dj 0J0 < 0}) > 

It is well known that either condition (12. 3p or conditions (I2.3P and 
(12 .4p together are needed in the proofs of maximum principles for sys- 
tems (see [5J, [S], [22] and the references therein). In particular if both 
are fulfilled the strong maximum principle holds as it is shown in the 
next theorem. 

Notation remark: here and in the sequel inequalities involving 
vectors should be understood to hold componentwise, e.g. if \1> = 
(■01, . . . , ip m ), ^ nonnegative means that ipj > for any index j = 
1, . . . ,m. 

THEOREM 2.1. (Strong Maximum Principle and Hopf's Lemma). 
Suppose that (|2Tl) . Q and (J23D hold andU = (u u . . . , w TO ) G C^fi; R 
is a weak solution of the inequality 

-AU + D(x)U > in fi 

i.e. 
(2.5) 



V[/-V^+ J D(z)([/,^) 



V«j ■ Vtfji + d i j(x)ui'ipj dx > 
.i=l *>i=i 

/or an?/ nonnegative $ = (?/>i, . . . , ?/> m ) G C^(fi; M m ). 
If U > in Q, then either U = in fi or U > in fi. In £/ie latter 
case if P E <9fi and J7(P) = then f^-(P) < ; where v is the unit 
exterior normal vector at P. 

Proof. It suffices to apply the scalar strong maximum principle using 
(12. 3 p to obtain that each component n& is either identically equal to 
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zero or positive, and then to use ( 12. 4p to prove that the same alternative 
holds for all the other components. To be more precise, for any equation 
we have — Auj + djjUj > Y^i^j ~^ii u i > 0? since < if % ^ j and 
Ui > 0. This implies that, for any j = 1, . . . , m, either Uj = or Uj > 
and in the latter case by Hopf 's Lemma we have the sign of the normal 
derivative on a point of the boundary where the function Uj vanishes. 
Suppose now that U does not vanish identically in Q and let J C 
{1, . . . , m} the set of indexes j such that Uj > in Q. Suppose by 
contradiction that J is a proper subset of {l,...,m}, and let / = 
{1, . . . , m} \ J the set of indexes % such that Ui = 0. If io, jo are as 
in (Ell) then -Au k] + d ioio u io > ~Y,j^i ~di j u j > ~di ,j u jo ^ 0, so 
that by the scalar strong maximum principle we get Ui > 0, which is 
a contradiction. □ 

Let us recall that weak maximum principles, Harnack inequalities 
and other estimates have been studied in many papers with general 
conditions and also for elliptic operators not in divergence form (see 
[22] and the references therein). 

We are interested in the quadratic form associated with system (12.21) . 
namely 



Q(tf ; f2)= / {\W\ 2 + D(x)(y,y)\dx = 
Jn 

(2.6) „ m m 

,i=i *ii=i 



dx 



for $ G Cl(n;R m ) ( or ^ G H^(Vl;R m ) ). 

It is easy to see that this quadratic form coincides with the quadratic 
form associated to the symmetric system 



(2.7) 



i.e. 



-AU + C(x)U = F in ft 
U = on dn 

-Aui + CuUi H h ci m u m = fi 

Au m -\- c m iUi -\- • • • -\- c mm u m f n 



where 



(2.8) C= l -(D + D l ) i.e. C=(c ij ), % = X - (d i5 + dp) 

So to study the sign of the quadratic form Q we can also use the prop- 
erties of the symmetric system. 

Therefore we review briefly the spectral theory for this kind of sim- 
metric systems, and use it to prove some results that we need for the 
possible nonsymmetric system ( 12. 2p . 
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Remark 2.1. If system ( 12.21) is cooperative, respectively fully coupled, 
so is the associate symmetric system (12. 7p . 

2.1. Spectral theory for symmetric systems. Let Q be a bounded 
domain in R , N > 2, and consider for m > 1 the Hilbert spaces 
L 2 = L 2 (ft) = (L 2 (Q)) m , a] = Hj(fi) = (^ 1 (^)) m , where if / = 
• • • > /m)> 9 1 — (fllj ■ ■ ■ > 9m) the scalar products are defined by 

m m „ 

i=l i=l ^ n 



(2.9) 



m m „ 

i=l i=l ^ Q 



Let C = C(x) = (cy(#))ij = i a symmetric matrix whose elements are 
bounded functions: 



(2.10) 

and consider the bilinear forms 
(2.11) 



a.e. in Q 



b(u,$)= [ \yu-v$ + c(u,$)]= [ 

Jn Jn 
and, for A > 

B A (U,$)= [ [VU • V$ + (C + AI) (U, <&)] 



(2.12) 



i=l 



Since c^- E L°° and Cjj = Cji, B and I? A are continuous symmetric 



V>i Vj I 



2 ), there exists 



bilinear forms, and, since | J n c 
A > such that B A is coercive in Hq, i.e it is an equivalent scalar 
product in H . 

By the Riesz representation theorem, identifying F E L 2 with the linear 
functional U E Hq h- > (U, F) L 2 , for any F E L 2 there exists a unique 
U =:TF E Hi such that \\U\\ H i < c||/|| L 2 and B A (U,$) = (F,U) h 2 
for any $ 6 H , i.e. Z7 is the unique weak solution of the system 



(2.13) 



i.e. 



-AU + (C(x) +M)U = F in Q 
U = onffi 

' -Aui + (en + A)ui H h ci m w m = fx in 

- Aw m + c mi Mi H h (c mm + A)w m = f m in Q 

«i = • • • = w m = on dQ 
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Moreover T : F i— )■ U, maps L 2 into L 2 and is compact because of 
the compact embedding of Hq in L 2 , it is a positive operator, since 
(TF, F) L 2 = (U, F) L 2 = B A {U, U) > if F / which implies U ^ 
(recall that 5 A is an equivalent scalar product in Hq ), and, since 
C is symmetric, it is also selfadjoint. Indeed if TF = U, TG = 
V, i.e B A {U,$) = (F,U)v, B A (V, $) = (G,V) h , for any $ G H*, 
then (TF,G) L2 = (C7,G) L2 = (G, *7) La = B A (V,U) = B A (U,V) = 
(F,V) L 2 = (F,TG) L 2. 

Thus, by the spectral theory of positive compact selfadjoint operators in 
Hilbert spaces there exist a nonincreasing sequence {nf} of eigenvalues 
with lim^oo fi A = and a corresponding sequence {W 3 } C Hq of eigen- 
vectors such that G(W 3 ) = /J, A W j . Putting X A = then Wj solves the 

systems -AW 3 + (C + A/)W j = X A W 3 , W j = on 90. Translat- 
ing, and denoting by Xj the differences Aj = X A — A, we conclude that 
there exist a sequence {Xj} of eigenvalues, with — oo < Ai < A 2 < . . . , 
lim^ +00 Xj = +oo, and a corresponding sequence of eigenfunctions 
{W J '} that weakly solve the systems 



(2.14) 



-AlfJ + CW 3 = XjW 3 in n 



W 3 = on <90 

i.e. if W 3 = (w u . . .,w m ) 

-Awi + c u wi + ■ ■ • + ci m w m = XjWi 



-Aw m + CmxWx H h C mm W m = XjW m 

Moreover by (scalar) elliptic regularity theory applied iteratively to 
each equation, the eigenfunctions W 3 belong at least to C 1 (0;M m ). 
We now collect in the next proposition the variational formulation and 
some properties of eigenvalues and eigenfunctions. 
In what follows if Q' is a subdomain of Q we denote by Afc(O') the 
eigenvalues of the same system with Q substituted by Q'. 

PROPOSITION 2.1. Suppose that C = (cy)£- =1 satisfies fl2TT0|) . 
and let {Xj}, {W 3 } be the sequences of eigenvalues and eigenfunctions 
that satisfy (jXHj) . 
Define the Rayleigh quotient 

(2.15) R{ y) = ^Xl forVeHl V^O 

with B(., .) as in (12.111) Then the following properties hold, where 
denotes a k-dimensional subspace of Hq and the orthogonality condi- 
tions VA-Wk or V-L Vfc stand for the orthogonality in L 2 . 

i) Ai = mm VeH i tV ^R(V) = mm VeB i j{ViV)L2=1 B(V,V) 
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ii) A m — » yeHj,yjto,Vii,..,OT m -i^r ) 

= mm v€Hl,(v,v) l2 =i,v±w 1 ,...y±w m - 1 B ( V i V ) if m > 2 

iii) A m = min Vm max VeVm ,v^oR(V) 

iv) A m = max Vm ^ mm v ±ev m -i ,v^oR{V) 

v) IfWe Hi, W ^ 0, and R(W) = A 1; i/ien W an eigenfunc- 
tion corresponding to \\. 

vi) lim meas ( n/)-40 Ai(fi') = +oo 

vii) // i/ie system is cooperative in Q and W is a first eigenfunction, 
then W + and W~ are eigenf unctions, if they do not vanish. 

viii) If the system is fully coupled in Q, then the first eigenfunction 
does not change sign in Q and the first eigenvalue is simple, 
i.e. up to scalar multiplication there is only one eigenfunction 
corresponding to the first eigenvalue. 

ix) Assume that the system is fully coupled in Vl, C = 

is another matrix that satisfies f)2.10p . and let {A'^}, be the 
sequence of eigenvalues of the corresponding system. Ifcij > c'ij 
for any i, j = 1, . . . , m then Ai > AV 

Proof. As before let us consider for A > the bilinear form B A (V, V) = 
B(V,V) + A(V, V) L 2, which is an equivalent scalar product in H , 
and define R A (V) = j0f} for V e H£ V ^ 0. Since R A (V) = 

R(V)+A, once the properties are proved for B\(V) (which is an equiva- 
lent scalar product in Hq) and its eigenvalues A^, we recover the results 
stated by translation. Therefore for simplicity of notations we assume 
from the beginning that A = i.e. that B(., .) is an equivalent scalar 
product in Hq. 

The proofs of i) ,. . . , vi) do not depend on cooperativeness and they 
are the same as the standard proofs in the scalar case (see [9], [12]), so 
we only sketch them. 

i) The sequence {Wj} is an orthonormal basis of L 2 , and since B(Wk, Wj) = 
Afc(M4,M / j) L 2 (in particular = if k ^ j), the sequence {(Aj)~2 Wj} 

is an orthonormal basis of Hq. It follows that if U = Y^kLi dkWj is the 
Fourier expansion of a function U in L 2 , the series converges to U in Hq 
as well. If now (U, U)v = ET=i d l = then B(U, u ) = Et X k d t > 
Ai J2k d-l = ^i an d i) follows. 

ii) If V±W U W m . x and {V, y) L 2 = 1), then V = ££° =m d ^W 3 and 
as before B(V, V) > X m and since B(W m , W m ) = X m ii) follows. 

iii) If dim (V m ) = m and {Vi, . . . , V m } is a basis of V m , there ex- 
ists a linear combination ^ V = YliLi a Ni which is orthogonal to 
Wi, ...W m -i (m coefficients and m — 1 unknown), so that by ii) 
we obtain that max vev m ,v^oR(V) > A m . On the other hand if 
V m = span (Wi,W m ) then max y e v m ,v^oR(V) < X m , so that iii) 
follows. 

iv) The proof is similar. If {Vi, . . . , V m -i} is a basis of an m — 1- 
dimensional subspace V m _x, there exists a linear combination ^ W = 
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YliLi a 'iWi °f the first m eigenfunctions which is orthogonal to V m _i, 
and R(W,W) < X m . So min y_L £ v m _i ,v^qR(V) < A m , but taking 
V m = span (Wi,W m _i) then min y±ev m _i , v^oR(V) > X m , so that 
iv) follows. 



By normalizing we can suppose that (W, W) L 2 = 1. Let V G Hq 



o- 



t > 0. Then by i) R(W + tV) 



> A x , i.e. B(W,W) + 



B(W+tV,W+tv) 
(W+tV)^ 

t 2 B(V, V) + 2tB(W, V) > Ai [(W, W) L 2 + t 2 (F, V) L 2 + 2t(W, V) L a] = 
Ai + \\^{V, V) + 2tX 1 (W, V). Since W) = A 1; dividing by t and 

letting t — > we obtain that V) > Xi(W,V) L 2 and changing V 

with -V we deduce that B(W, V) = X^W, F) L 2 for any V G Hj, i.e. 
W is a first eigenfunction. 
vi) If V G Hj(fi') there exists C > such that 

*0 > L I W l 2 rfx - C /n' l y l 2 dx > 
while by Poincare' s inequality 

/ n , l^l 2 ^ < C'lfi'l^^, |W| 2 dx, so that 



Vll 



Multiply ( EUD by = (u>+, 



1 



-C-»+ooif Ifi'l ->0. 

, and integrate. If in the 
z-th equation, multiplied by we write Wj = — w~ for j ^ % 
and recall that by cooperativeness —cijWjwf > 0, we deduce that 
B(W + , W + ) < Xi(W + , W + ), so that by v) W + is a first eigenfunction. 
The same applies to W~ 

viii) The conclusion follows from the strong maximum principle, which 
is valid under the fully coupling hypothesis. In fact if W + does not 
vanish, it is a first eigenfunction by vii), and by the strong maximum 
principle (Theorem 12. lj) is strictly positive in Q, i.e. W > in Q if it 
is positive somewhere. 

If W 1 , W 2 are two eigenfunctions corresponding to Ai, they do not 
change sign in f2, so that they can not be orthogonal in L 2 . This 
implies that the first eigenvalue is simple. 

ix) Let W 1 = (wi, . . . , w m ) the first eigenfunction for the system (12.141) . 
Since the system is fully coupled, W 1 does not change sign, and by 
normalizing it, we can assume that (W 1 , W 1 )^ = 1. 

Denoting by B' the bilinear form corresponding to the matrix C, since 
WiWj > and c^- > c'^ we get that 



(2.16) Ai = B{W l ,W 1 ) = [ 

Jn 



E i v --i 2 + E c ' 



|V^j| 2 + CijWiWj dx > 

dx = B'{W\W l ) > Xi 



ijWiWj 



1=1 



□ 
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2.2. Weak Maximum Principles for Cooperative systems. Let 

us turn back to the (possibly) nonsymmetric cooperative system (12. 2ft : 

-AU + D(x)U = F mil 
U = ondtt 
where the matrix D = (df,)™- =1 satisfies 

(2.17) dij G L°°{Q) , dij < , whenever % ^ j 

In the sequel we shall indicate by Xj = \( (— A + D; fi) the eigenval- 
ues of the associated symmetric system: 



(E2D 



—AU + C(x)U = F infi 
U = onffi 
where C = |(D + D*) i.e. = | (rf^ + dji). 

Analogously the corresponding eigenf unctions will be indicated by Wj . 
We also denote the bilinear form associated with the symmetric system 
dZBby 



B S (U,$)= [ [VU ■ W + C{U, $)] = /" 



J2 v«i -v^ + y^ 

_i=l i,j=l 

As already remarked, the quadratic form (12.61) associated to the system 
(12. 2p coincides with that associated to system (j2.7j) . i.e. 

Q(#;ft) = / |W| 2 + D(x)(V,V) = B s (^,y) 
Jn 

if * G /^(^R" 1 ). 

DEFINITION 2.2. We say that the maximum principle holds for the 
operator —A + D in an open set fi' C if any ?7 G H 1 (J7 / ) such that 

. [/ < on an' (i.e. C/+ G Hj(fi') ) 

• -AU + D(x)U < in fi' (i.e. / VC • V$ + D(x) (U, $) < for 
any nonnnegative $ G H ($7') ) 
satisfies J7 < a.e. in Q. 

Let us denote by \ { f(n') > the sequence of the eigenvalues of the 
symmetric system in an open set Q' C Q. 

THEOREM 2.2. [Sufficient conditions for weak maximum principle] 
Under the hypothesis Q2.17]L if \{ (O') > i/ien i/ie maximum princi- 
ple holds for —A + D m O' C O. 



Proof. By the variational characterization of eigenvalues 
AS s) = min yeH^o.vvo^ty) > 0, so that Q(V) = B S (V,V) > for 
any V ^ in H£(0')' Suppose that U < on 50', -At/"+D(x)Z7 < in 
fi'. Then, testing the equation with U + = (uf, . . . , writing in the 
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z-th equation Uj = Uj — u, for i ^ j, and recalling that —CijU^Uj > 
if i 7^ j, we obtain that B S (U + ,U + ) < 0, which implies U + = in 
ft'. □ 

Almost immediate consequences of the previous theorem are the fol- 
lowing "Classical" and "Small measure" forms of the weak maximum 
principle (see 0, [8], [20], [22]). 

COROLLARY 2.1. i) If (12. 3 p holds and D is a.e. nonnegative 
definite in fl' then the maximum principle holds for —A + D in 

ii) There exists 5 > 0, depending on D, such that for any sub- 
domain Q' C Q the maximum principle holds for —A + D in 
fl'Cfi provided \Q'\ < 5. 

Proof, i) If the matrix D is nonnegative definite then £? S (\I/,\E') > 
j n |V\I/| 2 , which implies that the first symmetric eigenvalue is positive, 
so that by Theorem 12.21 we get i) . 

ii) It is a consequence of the Poincare 's inequality through vi) of Propo- 
sition 12m □ 

Obviously the converse of Theorem 12.21 holds if D = C is symmetric: 
if the maximum principle holds for -A + C in Q! then > 0. 

In fact if < since the system is cooperative (and symmetric) 

there exists a nontrivial nonnegative first eigenfunction $i > 0, $ ^ 
0, and the maximum principle does not hold, since — A$i + C $i = 
Ai$i < in Q', $i = on dfl' , while $i > and $i ^ 0. 
However this is not true for general nonsymmetric systems. Roughly 
speaking the reason is that there is an equivalence between the validity 
of the maximum principle for the operator —A + D and the positivity 
of its principal eigenvalue Ax, whose definition is given below, and the 
inequality Ai(fi') > A^ wm ch can be strict, holds. 

More precisely we recall that the principal eigenvalue of the operator 
— A + D in an open set fl' C fl is defined as 

AiCtf) = sup{A € E : 3 * e Wf f(fi'; R m ) s.t. 

11 V > - A^ + D(x)V - A^ > in fl'} 

(see [5] and the references therein, and also [3] for the case of scalar 
equations) . 
We then have: 

PROPOSITION 2.2. Suppose that the system (J22H is fully coupled 
in an open set fl'Cf], Then: 

i) there exists a positive eigenfunction ^ e W^' c (JY; ]R OT ) which 
satisfies 

(2.19) 

- Atfi + D(x)# x = A^ft')*! w» O , fpOinfl', #i = on dfl' 
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Moreover the principal eigenvalue is simple, i.e. any function 
that satisfy (I2.19P must be a multiple of^>\. 

ii) the maximum principle holds for the operator —A + D in Q' if 
and only if Ai(O') > 

iii) if there exists a positive function \1/ G Wf^{Q!\ M. m ) such that 
$ > 0, -A* + D(x)* > in ft', then either Ai(fi') > or 
Ai(fi') = and ^ = c^i for some constant c. 

iv) Ai(fi') > A( (0') ; with equality if and only if^i is also the first 
eigenfunction of the symmetric operator —A + C in Q' , C = 
\{D + D l ). If this is the case the equality C{x)^\ = D(x)^i 
holds and, if m = 2, this implies that dyi = d 2 i- 

Proof. We refer to [5] for the proofs of i) - iii). For what concerns 
iv) we observe that evaluating the quadratic form Q(^] Q') on and 
recalling that it coincides with the quadratic form associated with the 
symmetric operator —A + C , C = \{D + D l ) we obtain that 

A^fi') = ggffj > \fiSl') with equality if and only if * x is the 
first symmetric eigenfunction, by Proposition 12. II v) . If this is the case, 
since satisfies the system (I2.19P and the system (12.21) . the equality 
D{x)^i = C(x)^?i follows. Since ^ is positive, if m = 2 the equality 
di2 = <i 2 i follows. □ 



3. Results on semilinear systems 

Let Obea bounded domain in R N , F = (f u . . . , f m ) :fix R m -»■ 
W 71 a C 1 '" function and consider the semilinear elliptic system for the 
unknown vector valued function U = (u\, . . . , u m ) in Q : 



(3.1) 



i.e. 



-AU = F(x, U) in Q 
U = onffl 

, -Au 1 = f 1 (x,u 1 ,...,u m ) mfl 



-Au m = f m (x, Ux,...,u m ) intt 
k Ui — ■ ■ ■ — u m — on dfl 

The system (II. ip is a particular case of this system, whith radial de- 
pendence on x. 

DEFINITION 3.1. • We say that the system (J3ZEJ is coopera- 
tive or weakly coupled in an open set Q' C Q if 



d Uj 



(x, U\, ... , u m ) > for every (x, U\, . . . , u m ) G Q' x 



and every i, j = 1, . . . ,m with i 7^ j. 
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• We say that the system (13.1 j) is fully coupled along a solution 
U in Q' C fi if it is cooperative in f2' and in addition V/, J C 
{1, . . . , m} such that / ^ 0, J ^ 0, In J = 0, JU J = {1, . . . , m} 
there exist « £ ^> jo £ ^ suc h that 

meas ({x G IT : -|^(x, J7(a;)) > 0}) > 
du jo 

Let us recall the following definition, which makes sense also in pos- 
sibly unbounded domains. 

DEFINITION 3.2. i) Let U be a C 2 (fi;M m ) solution of fl3~Tj) . 
We say that U is linearized stable (or that has zero Morse index) 
if the quadratic form 

Qu(*;to) = / [|W| 2 - J F (x,C/)(^,*)]dx = 
in 



(3.2) 



m 



,i=i *j=i 



9m 



> 



for any * = (^i, ... , V>m) G C£(fi; M m ) where J F (x, £7) is the 
jacobian matrix of F computed at U. 

ii) U has (linearized) Morse index equal to the integer // = /i(£7) > 
1 if /i is the maximal dimension of a subspace of Cl{VL\W n ) 
where the quadratic form is negative definite. 

iii) U has infinite (linearized) Morse index if for any integer k there 
is a A;- dimensional subspace of C^(Q; W 11 ) where the quadratic 
form is negative definite. 

As observed in Section 2, the quadratic form Qu associated to the 
linearized operator at a solution U, i.e. to the linear operator 

(3.3) Lu(V) = -AV - J F {x,U)V 

coincides with the quadratic form corresponding to the selfadjoint op- 
erator 

(3.4) Lu(V) = -AV - ~ (j F (x, U) + J F {x, U)) V 

where J F is the transpose of the matrix Jp. 

Hence if and W k denote the symmetric eigenvalues and eigen- 
functions of La, i.e. W k satisfy 

-AW k + CW k = X k W k in fi 

w k = o on9n, 

where C = cy(x), Cij(x) = -\ §l*:(x,U(x)) + §^-(x, U(x)) 
as in the scalar case we can prove the following 



SYMMETRY FOR COOPERATIVE ELLIPTIC SYSTEMS 



17 



PROPOSITION 3.1. Let tt be a bounded domain in Mr . Then the 
Morse index of a solution U to ( 13. ip equals the number of negative 
symmetric eigenvalues of the linearized operator Ljj. 

Proof. Let us denote by (jl(U) the Morse index as previously defined, 
and by m(U) the number of negative symmetric eigenvalues. 
If Qu is negative definite on a m-dimensional subspace of C*(f2;K m ), 
by Proposition 12.11 iii) the m-th symmetric eigenvalue X m is negative, 
so that m(U) > fi(U). 

On the other hand if there are m negative eigenvalues of the sym- 
metric operator Ly in Q, by the continuity of the eigenvalues there 
exists a subdomain Q' C Q where there are m negative eigenvalues and 
corresponding orthogonal eigenfunctions W 1 , . . . ,W m which by triv- 
ial extension can be considered as functions with compact support in 
Q. Regularizing these functions we get that the quadratic form Qu 
is negative definite on a subspace of C*(f2;IR m ) spanned by m linear 
independent functions, so that fi(U) > m{U). □ 



3.1. Preliminary results. Let e G S 1 be a direction, i.e. e G R , 
\e\ = 1, and let us define the hyperplane T(e) and the "cap" Q(e) as 

T(e) = {x G R N : x ■ e = 0} , Q{e) = {x G Q : x ■ e > 0} 

Moreover if x G Q let us denote by a e (x) the reflection of x through 
the hyperplane T(e) and by U ae the function U o a e . 



LEMMA 3.1. Assume that U is any solution of ( II. ip and £/iat the 
hypotheses i)-iii) of Theorem li.il hold. Then, for any direction e G 
S N ~ 1 ! the function W e = U — U <Je = (wi, . . . ,w m ) satisfies in fl(e) 
( and in VL) a linear system 

-AW + B e (x)W =0 mfi(e) 
W = on dfi(e) 



(3.5) 
i.e., i/5 e 



' -Awi + butoi H h 6i m ^m = m Q(e) 



-Aw m + 6 m i«r H h b mm w m = m O(e) 

«>i = • • • = w m = on 90(e) , 

siic/i t/iat /or any i, j = 1, . . . ,m and ieU t/ie following hold: 



(3.6) 



aw 



-(|ac|,l/(a;)) if u^x) > < e (x) 



5/ 



(3.7) bij(x) > --^(\x\,U(x)) if Ui(x) > u° e (x) , uj(x) > nj e (x) 
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while if Ui(x) = itf e (x), Uj(x) = u°j e (x) then hj(x) = —^-(\x\,U(x)). 
Moreover the system ( 13. 5 p is fully coupled along W e in Q(e). 

Proof. From the equation —AU = F(\x\,U(x)) we deduce that the 
reflected function U Ce satisfies the equation — AU Ue = F(\x\,U ae (x)) 
and hence the difference W e = U — U <7e satisfies 
-AW e = F(\x\,U) - F(\x\, U"*). 

We can write the first equation of this system as 

- Aw 1 = [f 1 (\x\,u 1 ,u 2 , ...,u m )- f 1 (\x\,u x %u 2 , ■ ■ -,u m )] + 

[fl(\x\,Ul%U 2 , ...,U m )- /iO|,<% u 2% • • -,«m)] + 

. . . [fcdxl <% u° 2 % . . . , u m ) - h{\x\, <% ul% U%)] 

Then we define, using iii): 




Hence we can write the equation as 

-Awi + bu(x)(ui - < e ) H h b lm (x)(u m - = 

Proceeding analogously for the other equations and letting i playing 
the same role of 1 in the first equation, we get that W e satisfies the 
linear system (13. 5 p whith B e = (&i 3 -)ij=i defined by 

(3.8) bu(x) = - I ^ Tp^ [\x\,tUj(x) + (1 - t)uj(x ae ),u k (x)} dt 
while if j 7^ 2 

(3.9) bij(x) = - [ ^[\x\,u^,t Uj (x) + (l-t)u^(x)]dt 

Jo v u j 

By ii) and iii), using Remark \1.2\ we get (I3.6p . (13. 7p . 

Obviously if Uj(x) = Ui(x ae ), Uj(x) = Uj(x ae ) then bij(x) = — §^(\x\, U( 

Since the system ( II. ip is weakly coupled in f2(e), Jj^ = > for 

any i ^ j, so that 6y < and the system (13. 5p is weakly coupled in 
Q(e) as well. 

The system (13. 5p is fully coupled in f2(e) because if /, J are as in 
Definition 13. 1\ there exist i G I, jo G J, x G f2(— e), such that 
|^-(|x|, t/(x)) = -^-(Ixl,^^),^^)) > 0, since by hypothesis i) 
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the system (11.11) is fully coupled along U in 0(— e). 
Then by the nonnegativity and continuity of 8 f' 0J0 also 

auj Q 

Kh (x ae ) = ~ [ u i0 (X) , tu j0 ) + ( 1 - t)u j0 (x)} dt 

Jo OUj 

is negative, with x" e G 0(e) . □ 

LEMMA 3.2. Let U = (ui, . . . , u m ) be a solution of (II. ip and assume 
that the hypothesis i) of Theorem \l.l\ holds. If for every e G S N ~ l we 
have either U > U Ue in 0(e) or U < U ae in 0(e), then U is foliated 
Schwarz symmetric. 

Proof. Let us start by proving that each component Ui of the solution 
is foliated Schwarz symmetric with respect to a vector pi G M , % = 
1, . . . , m. To do this we argue as in Lemma 2.4 in [11] . 
Let pi G S N ^ be such that for some r > S r = {x G ~R N : \x\ = r} C Q 
and Ui(rpi) = max5 r Mj. 

We define T+ = {e G 5 7V ~ 1 : e-p< > 0} and T~ = {e G S^" 1 : e-^ < 0}. 
To prove that is foliated Schwarz symmetric with respect to Pi it 
suffices to show that 

(3.10) Ui(x) > Ui((T e (x)) for all x G 0(e) 

whenever e G T+. 

Indeed this would imply the reverse inequality for e G 3^7, so that 
would be axially symmetric about the axis with direction pi, and the 
angular monotonicity would follow also by the same inequalities. 
So for fixed e G we consider the difference Wi = U{ — u^ e , and by 
assumption either Wi > 0, or W{ < 0. In the first case (|3.10|) . is satisfied. 
In the second case, since the system (13.51) is cooperative, we have that 
hj < if % 7^ j so that, since Wj < 0, 

—Awi + buWi = — bijWj < , Wi < in O(e) 

Then, by the (scalar) Strong Maximum Principle, we get that either 
Wi < or uii = in O(e), and the first case is not possible, since 
rpi G O(e) and Wi(rpi) > because Ui(rpi) is the maximum of Ui on 
S r . Hence w i = in O(e) and (13. IQj) is satisfied. 

At this point, to prove that the solution U is foliated Schwarz sym- 
metric we only need to prove that the vectors Pi are the same vector 
for all i — 1, . . . ,m. To this aim consider the vector p\ and take any 
hyperplane T(e) = {i£ : x • e = 0} passing through the axis with 
direction p\. The difference W e = U — f/ " 13 = (u>i, . . . , w m ) satisfies in 
O(e) the linear system (13. 5p . which is fully coupled along W e in O(e). 
Moreover either U > U ae in O(e) or U < U ae in O(e) by hypothesis, 
and W\ = in 0(e). So by Theorem 12.11 necessarily we have that iOj = 
in 0(e) for all j = 1, . . . , m, i.e. all vectors pj coincide with p\. □ 
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LEMMA 3.3. Let U = (ui, . . . , u m ) be a solution of ( II. ip and assume 
that the hypothesis i) of Theorem \l.l\ holds. 

Suppose that there exists a direction e such that U is symmetric with 
respect to T(e) and the principal eigenvalue Ai(fi(e)) of the linearized 
operator Ljj(V) = —AV — Jf(x, U)V in Q(e) is nonnegative. Then U 
is foliated Schwarz symmetric. 

Proof. We adapt to the case of systems the proof of Proposition 2.3 in 

After a rotation, we may assume that e = €2 = (0,1,... 0) so that 
T(e) = {i£ M. N : X2 = 0}. To prove the assertion we will use Lemma 
I3.2[ so we consider a direction e' G S^ -1 , e' 7^ e. After another orthog- 
onal transformation which leaves e 2 and T{e,2) invariant, we may as- 
sume that e' = (cos 6*0, sin^o, 0, . . . , 0), for some 9 G (— | , f ). Now we 
choose new coordinates, replacing (£1,22) by polar coordinates (r, 9), 
X\ = rcos9, %2 = r sm9, and leaving x = (X3, . . .xn) unchanged. 
The derivative U$ of U with respect to 9, extended in the origin with 
Ug(0) = if Q is a ball, satisfies the linearized system, i.e. 

(3.11) - AU e - J F (\x\,U)U e = in fi(e 2 ) 

Moreover, by the symmetry of U with respect to the hyperplane T{e^) 
we have that Ug is antisymmetric with respect to Tfa) and therefore 
vanishes on T(e?) and since it vanishes on dQ, it vanishes on dQfa) as 
well. 

Thus if Ai(f2(e)) > 0, since the maximum principle holds, we get that 
U = in fi(e). 

If instead Ai(fi(e)) = and f/g ^ 0, by the simplicity of the principal 
eigenvalue we get that Ug is a principal eigenfunction and hence it is 
positive (or negative) in Q(e). 

In any case Ug does not change sign in f2(e), and arguing exactly as in 
Proposition 2.3 of [TjJ] we get that for any direction e either U > U ae 
or U < U ae . Thus, by Lemma [3.21 U is foliated Schwarz symmetric. 

□ 

LEMMA 3.4. Assume that U is a solution of (11.11) and that the hy- 
potheses %)-%%%) of Theorem \l.l\ hold. Then 

Qu {(W e ) + ; 0(e)) = Qu {(U - U^) + ; fi(e)) < V e G S^ 1 
where Qu is the quadratic form defined in (I3.2p . 

Proof. As we saw in Lemma 13.11 the difference W e = U — U ae satisfies 
the linear system (13. 5p . Let us multiply the z-th equation of this system 
by (ui—u^ e ) + and integrate. By Lemma [37X1 we get, since bik < if i 7^ k 
and in the domain of integration where 6^ appears Ui(x) > u^ e (x), 
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u k {x) > < e (x) (so that b lk (x) > -§&(\x\,U(x))) : 
(3.12) 0= / (\V(u i -u°°) + \ 2 + b ii (x)[(u i -u°°) + ] 2 ) + 



E / - < e )1 [(«* - < e ) + - K - «?)-]) > 

(|v(^ - < e ) + | 2 + - o + ] 2 ) + 
E / -«f") + ] [K-< e )1)> 



V(«i - < e ) + | 2 - ^- L (|a;|,Mi,M2, • • .,u m )[(ui - < e ) + 

- E f (J^M' Wl ' U2 ' • • • ' - u i e ) + ( u k ~ ul e ) + ^j 

for any i = 1, . . . , m. Hence we obtain that 

(3.13) > Qu (K - . . . , (u m - + ; fi(e)) 

□ 

LEMMA 3.5. Suppose that U is a solution of { 1 1.1 1) tuii/i Morse index 
fjt(U) < N and assume that the hypothesis i) of Theorem li.il holds. 
Then there exists a direction e G S' Ar_1 sttc/i t/iat 

Qu(^;Q(e))>0 

for any % G Cl(tt(e);R m ) . 

Equivalently the first symmetric eigenvalue \*(L U , fi(e)) o/i/ie linearized 
operator Ljj{V) = —AV — Jf(x, U)V in fl(e) is nonnegative (and hence 
also the principal eigenvalue \i(Lu,Q(e)) is nonnegative). 

Proof. The proof is immediate if //(£/) < 1. In fact in this case for any 
direction e at least one amongst X\(L u ,fl(e)) and \\(L U ,Q(— e)) must 
be nonnegative, otherwise taking the corresponding first eigenfunctions 
we would obtain a 2-dimensional subspace of C^(fi; M m ) where the qua- 
dratic form is negative definite. 

So let us assume that 2 < j = fi(U) < N and let $i, . . . , $j be mutu- 
ally orthogonal eigenfunctions corresponding to the negative symmetric 
eigenvalues X\(L U , Q), . . . , X S j(L u , Q). 

For any e G S*^ -1 let $ e be the first positive I? normalized eigen- 
function of the symmetric system associated to the linearized operator 
Lu{V) in f2(e). We observe that $ e is uniquely determined by the 
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assumption i). Define 



(3.14) * e (x) = 




(<fr- e ,<frl) L 2 (n( _ e)) \ 
(*e,*l) L 2 (n(e)) J 

( (^-e,^l) L 2 (n( _ e 
I, (*e,*l) L 2 (n(e)) 




if x G 0(e) 



if x G 0(-e) 



The mapping e h> $ e is a continuous odd function from 5 
and, by construction, (\& e , $i)L 2 (n) = 0- 
Therefore the mapping h : S' Ar_1 — >• M J_1 defined by 



to H, 



o 



h(e) = ((* e , $ 2 ) L2(n) ,...,(* e , ^) L 2 (n) ) 



is an odd continuous mapping, and since j — 1 < N, by the Borsuk- 
Ulam Theorem it must have a zero. This means that there exists a 
direction e G S^" 1 such that \l/ e is orthogonal to all the eigenfunctions 
$i, . . . , $j. Since fi(U) = j this implies that Qui^e] 0) > 0, which in 
turn implies that either Qt/(<I> e ; 0(e)) > or Q[/($- e ; 0(— e)) > 0, i.e. 
either X\(L U , 0(e)) or A^(L„, 0(— e)) is nonnegative, so the assertion is 
proved. □ 

3.2. Proof of the symmetry results. 

Proof of Theorem By Lemma 13.51 there exists a direction e such 
that the first symmetric eigenvalue X\(L U , 0(e)) of the linearized oper- 
ator is nonnegative, so that the principal eigenvalue Ai(0(e)) is nonneg- 
ative as well. Moreover by Lemma [33] we have that Qu {(U — U ae ) + ) < 
0, so that either (U - U Ue ) + = 0, or X\(L U , 0(e)) = and (U - U Ue ) + 
is the positive first symmetric eigenfunction in O(e). In any case we 
have that either U < U ae or U > U ae in O(e) holds. 
If U = U a " in O(e) then by Lemma [3.31 since the principal eigenvalue 
Ai(0(e)) is nonnegative, we immediately obtain that the solution U is 
foliated Schwarz symmetric. If this is not the case and e.g. U > U ae 
then, by Lemma [3.11 we have that U > U ae in 0(e). 
We now apply, as in [19] and [llj . the "rotating plane method", which 
is an adaptation of the Moving Plane method as developed in [2] and 
obtain a different direction e' such that U is symmetric with respect 
to T(e') and the principal eigenvalue X\(L U , 0(e')) of the linearized op- 
erator in O(e') is nonnegative. Then by Lemma 13.31 we get that U is 
foliated Schwarz symmetric. 

More precisely, without loss of generality we suppose that e = (0, 0, . . . , 1) 
and for 9 > we set eg = (sin 6*, 0, . . . , cos6>), so that e = e, and 
Q e = 0(e e ), U e = U a «e , W 6 = U - U a *e , Let us define 9 = sup{# G 
[0,7r) : U > U 9 in Qg}. Then necessarily # < ^, since (U — U°)(x) = 
—(U — U 7T )(a e7I (x)) for any x G O (and cr eir (x)) G 
Suppose by contradiction that U ^ U e ° in 0^ . Then, by Lemma 13. ip 
applied to the difference W e ° = (U - U 00 ), we get that W e ° > in 
Ofl . Taking a compact K C Oq whith small measure and such that 
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(componentwise) W 9 ° > (rj, . . . , rj) for some r] > 0, for 9 close to 9 we 
still have that W e > (f , . . . , §) in while W e > in O e \ K by the 
weak maximum principle in domains with small measure. This implies 
that for 9 greater than and close to 9 the inequality U > U e in Qg still 
holds, contradicting the definition of 9$. 
Therefore U = U e ° in Qg . 

Observe that the difference W 9 satisfies the linear system ( 13. 5 p and 
does not change sign for any 9 G [0,6*o), which implies by Proposition 
12.21 that it is the principal eigenfunction for the system ( 13. 5 p corre- 
sponding to the eigenvalue Ai = 0. As 9 — > 9 a the system (13. 5p tends 
to the linearized system, in the sense that the coefficients bij (x) tend to 
the derivatives ^-(\x\,U(x)). Then by continuity the principal eigen- 
value Ai(f2( e o )) °f the linearized operator Lu(V) = — AV — J F (x, U)V 
in fl(eg ) is zero. □ 

Proof of Theorem \l. 6 A Let us choose N orthogonal directions e\, e%, . . . ejy 
S N ~ X and introduce new (cylinder) coordinates (r, 9, 7/3, . . . , x/n) defined 
by the relations x = r[cos6*ei + sin0e 2 ] + J2iL 3 yi^i- 
Then the angular derivative Ug of U with respect to 9, extended by 
zero at the origin if SI is a ball, satisfies the linearized system, i.e. 



(3.15) 




J F {\x\,U)U e =0 infi 
= ondQ 



By the stability assumption we have that the first symmetric eigen- 
value of the linearized system in Q is nonnegative, so that by iv) of 
Proposition 12.21 also the principal eigenvalue Ai(fi) is nonnegative. 

Then by (13.151) and Proposition 12.21 we get that U$, if does not van- 
ish, must be the first symmetric eigenfunction corresponding to the 
eigenvalue and hence it does not change sign in Q. Since Ug is 2-k- 
periodic this is impossible and therefore Ug = 0. By the arbitrarity of 
the vectors ex, e<i we conclude that U is radial. □ 

Proof of Corollary As remarked at the beginning of the proof of 
Lemma 13. 5[ if U is a Morse index one solution for any direction e 
either Af(L w , 0(e)) or X\(L U ,Q(— e)) must be nonnegative. By the 
proof of Theorem 11.11 we can find a direction e such that U is sym- 
metric with respect to the hyperplane T(e) and the principal eigen- 
value Ai(0(e)) = Ai(0(— e)) > 0. On the other hand by the symme- 
try A s 1 (0(e)) = A s 1 (0(-e)) > 0. If Ai(0(e)) > then, by the maxi- 
mum principle, the derivatives Ug as defined in the proof of Lemma 
I3.3[ must vanish , and hence U is radial. So if Ug ^ necessarily 
A! (0(e)) = A s 1 (0(e)) = and by v) of Proposition f272l Ug is the first 
eigenfunction of the simmetrized system, as well as a solution of (13.151) . 
So we get that J F (\x\,U)Ug = \ (J F (\x\,U) + J F (\x\,U)) Ug, i.e. ( TOj) 
and if m = 2, since Ug is positive, we get (II. 4p . □ 
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4. Examples and comments 

A first type of elliptic systems that could be considered are those 
of "gradient type" (see [7]), i.e. systems of the type ( ll.ip where 
fj(\x\,U) = J^-(|x|,£7) for some scalar function g G C 2,a ([0,+oo) x 
W m ). 

In this case the solutions correspond to critical points of the functional 

$( W ) = I I \VU\ 2 dx- [ g{\x\,U)dx 
2 Jn Jn 

in Hq(Q) and the linearized operator (13. 3p coincides with the second 
derivative of $. 

Thus standard variational methods apply which often give solutions 
of finite (linearized) Morse index, as, for example, in the case when the 
Mountain Pass Theorem can be used. So, if the hypotheses of Theorem 
11.11 are satisfied, our symmetry results can be applied. 

Many systems of this type have been studied (see [7J and the refer- 
ences therein). An example is the nonlinear Schrodinger system 



(4.1) 



— Aui + ui = \ui\ 2q 2 ui + b\u 2 \ q \u\\ q 2 ui in Vt 
-Au 2 + u 2 u 2 = \u 2 \ 2q ~ 2 u 2 + b\ui\ q \u 2 \ q ~ 2 ui in Q 



where b > 0, is either a bounded domain or the whole R , N > 2, 
q > 1 if iV = 2 and 1 < q < jf^ if iV > 3. If SI ^ R N the Dirichlet 
boundary conditions are imposed 

(4.2) ui = u 2 = on dQ 

This type of system has been studied in several recent papers (see PQ, 
[T4] and the references therein). 

It is easy to see that the system is of gradient type and the solutions 
of ( I4.ip . ( 14. 2 p are critical points of the functional 
(4.3) 

/([/) = I( Ul ,u 2 ) 



- [ \VU\ 2 dx-— f (\ Ul \ 2q +\u 2 \ 2q ) dx-- [ \ Ul u 2 \ q dx 
2 Jn 2g J Q q J n 



in the space Hg(O). 

Thus the linearized operator ( 13. 3 p at a solution U corresponds to 
the second derivative of I in Hq(Q) and hence the corresponding linear 
system is symmetric. 

Moreover, as observed in [14] , the system (14. ip is cooperative, and 
fully coupled in f2(e), for any e G S N ^, along every purely vector 
solution U = (tii, u 2 ), i.e. such that u\ and u 2 are both not identically 
zero. In particular the system is fully coupled along positive solutions 
in any Q(e). 

Then Theorem 11.11 could be applied to any purely vector solution with 
Morse index less than or equal to N, in an annulus or in a ball. In 
particular we can consider solutions obtained by the Mountain Pass 
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Theorem which have Morse index not bigger than one which are purely 
vector positive solutions for suitable values of b and q > 2, as could be 
obtained by the same proof of [H] for the case Q = M> N . 

A second type of interesting systems of two equations are the so 
called "Hamiltonian type" systems (see [7J and the references therein). 
More precisely we consider the system 



(4.4) 



with 
(4.5) 

dH dH 

/ 1 (|x|,«l,M 2 ) = (\x\,U X ,U2) , / 2 (M,Wl, «2) = TT~ (\x\,Ui,U 2 ) 

OU2 OUi 

for some scalar function H G C 2,a ([0,+oo) x R 2 ). These systems can 
be studied by considering the associated functional 

(4.6) J(U) — I(ui, U2) — — I Vu\-Vu 2 dx— I H(\x\, u\, 1*2) dx 

'n Jq 




2 



either in H (fl) or in other suitable Sobolev spaces (see [7]). 

It is easy to see that the linearized operator defined in (13. 3p does 
not correspond to the second derivative of the functional J, which 
is strongly indefinite. Nevertheless solutions of (|4.4j) can have finite 
linearized Morse index as we show with a few simple examples. 

Let us consider the following system: 

{-Aui = Ae" 2 in ft 
-Au 2 = fie ui in Q 
ui = u 2 = on dVt 

where A, \x G M. 

If A, /i > then the system is fully coupled along every solution in any 
ft(e), e G S 1 ^ 1 , and the hypotheses of Theorem 11.11 are satisfied. 
Let us consider the function G : R 2 x (C 2 > Q (ft)) 2 -»■ (C°- Q (ft)) 2 defined 
by G((X, fj), (ui, u 2 )) = (-Ami - Ae" 2 , —Au 2 - Xe Ul ). 
We have that G((0,0), (0,0)) = (0,0), ^g^((0, 0), (0, 0) )(</>, ^) = 
(— A0, — A^), and the first (symmetric) eigenvalue of this operator is 
strictly positive, so that the operator is invertible. 
By the implicit function theorem for small A, [i there is a unique non- 
trivial solution (ui(X, fj,), u 2 (X, fj,)) close to the trivial solution of the 
system (14. 7p corresponding to A = /i = 0. Moreover the solution is 
positive by the maximum principle and it is linearized stable, so that 
it is radial. Indeed the first (symmetric) eigenvalue of the linearized 
operator corresponding to A = /i = 0, -u = i> = 0is strictly positive, 
and by continuity it is positive for small A, 
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The same happens substituting the exponential with other nonlin- 
earities f(u 2 ), g(ui) with nonnegative derivative in a neighborhood of 
and such that f{0),g{0) > 0. 



We now consider, as another example, the system 
(4.8) 



-Aui = i£ in Q 



"2 



— Au2 = u\ in Q 
Mi, U2 > in Q 
ui = ii2 = on dQ 



where 1 < p,q < The idea is to proceed as in the previous 

example starting from the case p = q and the solution U\ = u 2 = z, 
where z is a scalar solution of the equation 

= z p in Q 

(4.9) ( > in Q 

= on 90 

which is nondegenerate. 

Let us observe that if p — q and z has Morse index equal to the 
integer fi(z), then fj,(z) is also the Morse index of the solution U = 
(ui = u 2 ) = {z,z) of the system (14.81) . Indeed the linearized equation 
at z for the equation (14.91) and the linearized system at (z, z) for the 
system (14. 8p are respectively 




(4.10) 
and 



-A<f> - pz p ' l (j) = in n 
= on dQ 




' -A0i - pz p - l (f)2 = in Q in Q 
( 1.1 1 ) { - A0 2 - pz p ~ Vi = in Q in tt 

= 0ondn 

This implies that the eigenvalues of these two operators are the same, 
since if (p is an eigenfunction for ( I4.10p corresponding to the eigen- 
value Afc then taking (j>i = (j) 2 = (f> we obtain an eigenfunction {4>i,(t>2) 
for ( 14. lip corresponding to the same eigenvalue, while if (4>i,<fi 2 ) is 
an eigenfunction for ( 14. lip corresponding to the eigenvalue then 
4> = 0i + 4>2 is an eigenfunction for (14. 10p corresponding to the same 
eigenvalue. 

So proceeding as in the previous example we can start from a non- 
degenerate solution of ( 14. 9 p with a fixed exponent p G (l,i^z§) and 
find a branch of solutions of (14. 8 j) corresponding to (possibly different) 
exponents p, q close to p. For example if we start with a Mountain Pass 
(positive) solution z in the ball of equation (14. 9 p with the exponent p, 
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knowing that its Morse index is one and it is nondegenerate, we get a 
branch of Morse index one radial solutions for p, q close to p. 

If Q is an annulus then a mountain pass solution z of (14. 9 p is foliated 
Schwarz symmetric, as shown in [17]. Then, if not radial, it is obviously 
degenerate. Nevertheless, working in spaces of symmetric functions, 
we could remove the degeneracy and apply the continuation method 
described above. 

Thus, starting from an exponent p for which the mountain pass so- 
lution z of (14 .9p is not radial, we can construct solutions U = (ni,w 2 ) 
of (14.81) in correspondence of exponents p, q close to p, with Morse in- 
dex one. Then Theorem 11.11 applies and, in particular, we get that the 
coupling condition ( II .4p holds, which in this case can be written as 

(4.12) pu p 2 ~ l = qu\~ l in VL 

Note that more generally, by Corollary 11.1] the equality ( I4.12p must 
hold for every solution U = (ui,U2) of (14. 8 p with Morse index one 
giving so a sharp condition to be satisfied by the components of a 
solution of this type. 

Let us remark that our results apply also when the nonlinearity de- 
pends on |x| (in any way). Arguing as before it is not difficult to 
construct systems having solutions with low Morse index, in particular 
with Morse index one. 

An example could be the "Henon system" 



(4.13) 



—Aui = \x\ a U2 in Q 

— Au 2 = \xfu\ in Q 

Mi, v-2 > in Q 

^Ui = u = on dQ 



with a, ft > 0, p, q > 1. 

Starting again from a solution of the scalar equation of mountain 
pass type it is possible to construct solutions of ( I4.13|) with the same 
Morse index for p, q and a, f3 close to each other. Note that, for some 
values of a, j3, p, q, such solutions would not be radial, even if Q is a 
ball, but rather foliated Schwarz symmetric as for the scalar case. 

Finally, as in the scalar case, we could consider nonhomogeneous 
equations and provide examples of convex nonlinearities for which there 
exist solutions of Morse index one which change sign and to which our 
results apply. 
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